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THE  EFFECT  OF  INCREASING  THE  DIFFICULTY  OF  THE 
RPA  TEST  OF  NIGHT  VISION 


SUMMARY 


Results  obtained  on  a  group  of  446  men  given  both 
the  standard  Radium  Plaque  Adaptometer  test  and  also 
the  same  test  at  a  brightness  ,20  log  units  lower  than 
the  standard,  indicate  that  a  much  larger  percentage 
of  men  will  necessarily  fail  the  test  if  the  bright¬ 
ness  is  lowered.  Classification  based  on  a  more 
difficult  test  has  the  same  statistical  properties  as 
that  based  on  the  regular  tost,  and  consequently  will 
doubtless  bo  no  more  reliable. 
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Among  the  several  suggestions  which  have  been 
advanced  for  the  improvement  of  the  Radium  Plaque 
Adaptometer  test  of  night  vision,  one  of  the  most 
common  has  been  the  addition  of  one  or  more  lower 
brightness  levels  to  the  present  single-brightness- 
level  test.  An  experiment  has  been  performed  to 
test  the  effect  of  such  a  modification. 

Experimental  Procedure: 

Two  Radium  Plaque  Adaptometers  were  fitted  with 
easily  manipulated  extra  "neutral”  filters  of  scotopic 
density  .20  plus  minus  ,02  log  units.  Each  man  of  a 
group  of  219  (Group  A)  was  tested  first  on  the  standard 
plaque  by  the  regular  procedure.  Immediately  there¬ 
after,  the  test  was  repeated,  with  the  extra  filter  in 
place.  The  procedure  was  not  altered  in  any  other 
manner,  and  the  subjects  were  not  informed  that  they 
were  receiving  any  but  the  ordinary  test. 

Another  group  of  227  men  (Group  B)  was  administered 
the  more  difficult  test  first,  with  the  standard  test 
immediately  following.  This  group  served  as  a  check  on 
the  possible  effect  of  practice  upon  the  RPA  scores, 
made  in  the  present  experiment,  and  as  a  source  of  in¬ 
formation  on  "encouragement"  or  "discouragement"  arising 
from  taking  a  test  of  greater  or  lesser  difficulty  first. 

All  tests  were  performed  by  three  Qualified  RPA 
Operators, 

Results! 

Results  are  reported  in  terms  of  Level  I  (regular 
RPA  testing  level,  approximately  3.9  log  uul)  and  Level 
II  (regular  RPA  with  extra  ,20  filter  added,  approx¬ 
imately  3.7  log  uul). 


Curry i lative  Frequencies : 

Tables  IA  and  IB  present  the  distribution  of 
scores  made  by  Group  A  on  both  testing  levels,  to¬ 
gether  with  cumulative  frequencies  and  cumulative 
percentages.  Corresponding  data  for  Group  B  are  given 
in  Tables  IIA  and  IIB.  Tables  II I A  and  IIIB  present 
percentages  failing  in  each  group,  and  corresponding 
Chi-Squares  on  each  of  the  testing  levels. 

Figure  1  is  a  graphical  presentation  of  cumu¬ 
lative  frequency  data.  These  data  in  an  identical 
plot  on  Probability  Paper  are  presented  in  Figure  lb, 
which  show  by  tho  reetilinearity  of  the  curve  fitted 
to  the  data  that  the  distribution  of  RPA  scores 
follows  a  normal  distribution.  The  parallelisim  of 
the  tvro  curves  indicates  that  the  variability  is  the 
same  on  both  testing  levels.  Deviations  at  the  upper 
ends  of  the  curves  from  the  fitted  straight-lines  are 
the  result  of  the  small  number  of  cases  represented  by 
each  point. 

These  data  indicate  that  Groups  A  and  B  are 
adequate  samples  of  the  general  population;  there  is 
no  significant  difference  between  their  performance 
on  either  level.  The  differences  in  the  percentages 
of  failures  are  within  the  error  of  sampling,  and  no 
systematic  differences  in  performance  which  may  be 
attributed  to  the  variation  in  the  order  of  ad¬ 
ministration  are  evident. 

Comparison  of  Performance  on  Levels  II  and  III. 

Scatter  plots  of  scores  made  on  Level  I  against 
scores  made  on  Level  II  are  given  in  Tables  IV  and  V 
for  Groups  A  and  B  respectively,  together  with 
approximate  tetrachoric  rls  (cosine  formula)*  about 
the  pass-fail  cut.  These  results  are  summarized  for 
the  combined  groups  in  Table  VI  A  and  B. 

The  performance  of  both  groups  on  the  test  at 
Level  II  differs  significantly  from  that  on  Level  I. 
Level  II  is  unequivocally  a  more  difficult  tort. 

*  More  accurate  determinations  of  this  value  were  not 
possible,  owing  to  the  small  number  of  cases  in  one 
cell  of  the  double  dichotony  which  necessitated  a 
correction. 
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It  should  be  noted  that  of  the  69  men  of  both 
groups  who  failed  the  regular  RPA  test  (Level  I), 
only  one  passes  the  test  at  Level  II.  The  con¬ 
ventional  cut-off  fails  an  inferior  group,  without 
equivocation.  Tetrachoric  r> s  (cosine  formula)  com¬ 
pare  not  unfavorably  y: it h  others  reported  on  tests 
of  night  vision,  and  conform  with  statistical  ex¬ 
pectancy. 

Selection  of  Superior  and  Inferior  Groups: 

Table  VIIA,  9,  and  C  presents  the  cumulative 
frequency  and  percentage  of  sc'ores  made  on  Level  II 
for  each  of  the  following  sub-groups  of  the  whole 
group  of  446  mens  Men  making  10/10  on  Level  I,  men 
making  from  16/20  to  I9/2O  inclusive  on  Level  I,  and 
men  failing  on  Level  I.  Figure  2a  presents  these 
data  graphically,  and  permits  the  estimation  of  the 
median  and  standard  deviation  of  scores  made  by  each 
sub-group.  These  are  included  in  Table  VTIIA.  The 
chi-square  test  of  the  hypo thesis  that  each  of  these 
sub-groups  is  a  representative  sample  of  the  whole 
group  is  given  in  Table  VIIIB.  Figure  2b  is  a  re¬ 
plot  of  Figure  2a  on  Probability  Paper.  Again  we 
are  dealing  with  normally  distributed  scares,  and 
with  the  same  variability. 

Values  of  chi-square  indicate  that  performance 
at  Level  I  predicts  p erf -nuance  on  Level  II  with 
statistical  significance.  The  median  scores  of  the 
sub-groups  differ  by  almost  one  standard  deviation, 
a  difference  which  is  significant.  This  finding 
suggests  that  those  making  10/10  on  the  RPA  test  may 
be  classed  ''superior'1.  This  is  very  probably  a 
procedure  of  dubious  merit,  however,  since  the  validity 
of  the  RPA  is  undetermined. 
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Discussion: 


In  previous  reports*,  the  statistical  basis  of 
the  RPA  test  of  night  vision  has  been  presented.  A 
test  of  this  type  must  be  based  upon  the  statistical 
properties  of  small  samples.  A  sample  of  20  trials 
provides  an  estimate  of  the  individual’s  probability  of 
seeing  at  a  fixed  brightness  level.  The  limits  of 
accuracy  are  such  that  all  individuals  whose  "true” 
probability  of  seeing  is  0$  will  score  11/20  and  below 
at  the  .001  level  of  confidence.  In  order  to  fail, 
those  whose  probability  of  seeing  is  30$  or  less  at  the 
same  level  of  confidence,  a  failing  score  of  15/20  is 
required.  These  statements  hold  irrespective  of  the 
brightness  level  at  which  the  test  may  be  run.  Any 
Pass-Fail  cutting  score,  then,  if  it  is  to  be  mean¬ 
ingful,  must  be  based  not  on  the  percentage  of  men 
it  is  desired  to  classify  as  FAIL,  but  on  the  statis¬ 
tical  probability  of  a  given  test  score.  The  percentage 
of  men  failed  must  be  manipulated  by  variation  of  the 
brightness  level  at  which  the  test  is  administered. 

In  the  present  experiment,  in  which  the  difficulty 
of  the  test  was  increased  by  lowering  the  brightness  of 
the  test  level  by  .20  l.u.,  the  percentage  of  men  fail¬ 
ing,  if  the  30$  frequency  of  seeing  is  desired  4s  a  cut¬ 
ting  performance,  is  radically  increased  from  15.5$  to 
59.9$.  If  it  is  desired  only  to  eliminate  those  whose 
frequency  of  seeing  Is  0$,  the  percentage  of  failures 
must  be  increased  from  4.9  to  28.8$  in  order  to  insure 
that  no  such  man  can  pass. 

The  present  testing  brightness  was  selected  be¬ 
fore  these  considerations  were  apparent.  It  has  proven 
satisfactory,  however,  and  has  been  used  since  the  test 
was  first  designed. 


*  Report  oOkfeJIufidred  Subj^ts T  e s t e d.anj . gstgst 
for  Fifty  Trials.  Each  on  the  Navy  Radium  Plaque 
Adantometer .  Research  Division,  Bureau  of  Medicine 
and  Surgery,  Washington,  D.  C.,  of  8  March  1944. 
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The  same  statistical  properties  of  the  test 
sharply  limit  the  correlation  which  may  be  found 
between  performances  on  two  levels  of  difficulty  of 
the  same  test,  and  the  tost-retest  correlation  as  well. 
The  present  data  confirm  the  first  expectation.  No 
direct  data  on  test-retest  reliability  are  presented, 
but  it  may  legitmately  be  inferred  that  it  will  behave 
as  expected. 

In  view  of  these  considerations,  it  may  be 
asserted  that  the  reliability  of  the  RPA  test  at  a 
lower  brightness  level  will  be  the  same  as  that  at 
the  present  level.  Nor  will  the  validity  of  the  test 
be  altered. 

Conclusions: 

1.  The  effect  of  setting  up  a  more  difficult  test 
by  fitting  the  RPA  with  an  additional  filter  of 
scotopic  density  .20  plus  minus .02  log  units,  is  to 
produce  a  test  which  a  larger  number  of  men  will 
necessarily  fail.  This  result  is  of  doubtful  de¬ 
sirability  in  a  test  of  undetermined  validity. 

2.  The  regular  RPA  test,  as  well  as  the  more 
difficult  test,  gives  scores  failing  a  normal 
distribution. 

3.  The  variability  of  the  population  is  the  same  in 
both  tests. 

4.  It  is  possible  to  identify,  on  the  basis  of  the 
regular  RPA  scores,  three  groups  of  different  per¬ 
formances  in  the  more  difficult  test.  Then  scores 
of  these  sub-groups,  who  may  be  called  "superior11, 
average"  and  "inferior"  are  normally  distributed,  and 
the  variability  of  each  group  averages  the  same 
magnitude.  Practical  application  of  this  finding 
would  be  unwise,  since  the  RPA  is  a  test  of  unde¬ 
termined  validity. 

5.  No  significant  purpose  will  be  served  by  alter¬ 
ing  the  difficulty  of  the  RPA  tost  of  night  vision. 

To  make  the  test  more  difficult  would  have  the  result 
that  a  larger  percentage  of  the  population  fail. 

This  is  of  doubtful  desirability  in  a  test  of  unde¬ 
termined  validity. 
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TABIE  IA 


TABLE  IB 


Group  A 


Distribution  of  Scores  on 
Level  I  (Standard  RFA  TEST) 


Score 

F 

CF 

_ C# 

10/10 

132 

132 

60 

19/20 

14 

146 

67 

18/20 

13 

159 

73 

I.7/2O 

19 

178 

81 

16/20 

13 

191 

87 

15/20 

7 

198 

90 

14/20 

8 

206 

94 

13/20 

3 

209 

93 

12/20 

4 

213 

97 

11/20 

1 

2H 

90 

10/20 

0 

214 

98 

9/20 

0 

214 

98 

8/20 

0 

214 

98 

7/20 

1 

215 

96 

6/20 

2 

217 

99 

5/20 

1 

218 

99 

4/20 

1 

219 

100 

Distribution  of  Scores  on 
Level  II  (Extra  ,20  l.u  Filter) 


Score 

F 

CF 

c$ 

10/3.0 

41 

41 

19 

19/20 

7 

48 

22 

18/20 

8 

56 

25 

17/20 

15 

71 

32 

16/20 

18 

89 

41 

15/20 

IS 

107 

49 

14/20 

10 

117 

53 

13/20 

15 

132 

60 

12/20 

21 

153 

70 

11/20 

9 

162 

74 

10/20 

15 

177 

81 

9/20 

12 

I89 

86 

8/20 

10 

199 

91 

7/20 

8 

207 

94 

6/20 

H 

i 

214 

98 

5/20 

2 

a6 

99 

4/20 

3 

219 

100 

F  —  Frequency 
CF  —  Cumulative  Frequency 
C$  —  Cumulative  Percentage 


TABLE  IIA 


TABLE  IIB 


'^oup  B 


Distin 

i.bution 

of  Score 

a  on 

Dis tribution 

of  Scor< 

js  on 

Level 

I  (Standard  RPA  Test) 

Level  II  (Extra 

.20  l.u 

Filter) 

Score 

F 

CF 

C% 

Score 

F 

CF 

_ J# 

10/10 

123 

123 

45 

10/10 

40 

40 

18 

19/20 

16 

139 

61 

19/20 

4 

44 

20 

18/20 

15 

154 

68 

18/20 

10 

54 

24 

17/20 

19 

173 

76 

17/20 

17 

71 

31 

16/20 

13 

186 

82 

16/20 

19 

90 

39 

15/20 

11 

197 

87 

15/20 

27 

117 

50 

14/20 

5 

202 

89 

14/20 

20 

137 

59 

13/20 

3 

205 

90 

13/20 

21 

158 

68 

12/20 

6 

211 

95 

12/20 

11 

169 

73 

11/20 

4 

2.15 

95 

11/20 

20 

I89 

82 

10/20 

8 

223 

99 

10/20 

12 

201 

87 

9/20 

2 

225 

99 

9/20 

8 

209 

90 

8/20 

1 

226 

99 

8/20 

6 

215 

93 

7/20 

1 

227 

100 

7/20 

1 

216 

93 

6/20 

4 

220 

95 

5/20 

3 

223 

96 

4/20 

1 

224 

96 

3/20 

2 

226 

98 

2/20 

1 

227 

100 

F  —  Frequency 
CF  —  Cumulative  Frequency 
C%  —  ■  Cumulative  Percentage 


TABLE  III 


Chi  Square  Test  of  Homogenity  of 
Ken  in  Group  A  and  Group  B. 

A 

Level  I 


Group  A  Group  B 

Obtained  Predicted* 

FASS  191  198 

FAIL  28  41  29 

Chi-Square  »  1,45 

The  Chi-Square  test  shows  that  the  chances 
that  the  performance  of  Group  B  on  Level  I  differs 
from  that  of  Group  A  are  greater  than  20,  and  less 
than  -30  in  100,  The  groups,  then,  constitute two 
samples  of  the  same  homogenous  population. 


B 

Level  II 


Group  A 

PASS  89 

FAIL  130 


Group  B 

Obtained  Predict ed* 
90  '  92 

137  135 


Chi-Square  =  0,10 

The  Chi-Square  test  shows  that  the  chances 
that  the  performance  of  Group  B  on  Level  II  differs 
from  that  of  Group  A  are  greater  than  50,  and  less 
than  95  in  100,  The  groups,  then,  constitute  two 
samples  of  the  same  homogenous  population. 


*  From  results  of  Group  A 


Ovnlailt*  FtronWn 


Cumilativ*  ?erc*ntf»g« 


Scatter  plot  —  Group  A 
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All  totrachoric  r's  (cosine  formula)  are  approximate,  a  correction 
being  necessary  for  the  small  number  of  cases  in  one  quadrant. 


Scatter  Plot  -  Group  3 
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TABLE  VI  A 


PASS-FAIL  Double  Dichotomy 
Level  I  vs.  Level  II 
(Groups  A  and  B  Grouped) 


PASS 

FAIL 

TOTAL 

(Level  I) 

(Level  I) 

PASS 

(Level  II) 

178 

1 

179  (40.1%) 

FAIL 

(Level  II) 

199 

68 

267  (59.9%) 

Plus  filter 

TOTAL 

377 

69 

446 

(84.5 %) 

(15.5%) 

Tetrachoric  r  s 
(cosine  formula) 

,76  (approx.) 

TABLE  VI  B 

Chi-Square  Test  of 

Homogeneity  of  Data 

obtained  on  Level  I  and  Level 

II 

Level  I 

Level  II 

Obtained 

Obtained 

Predicted 

PASS 

377 

179 

377 

FAIL 

69 

267 

69 

Chi-Square  ~  208 

Chi-Square  test  shows  that  the  chances  are 
infinitesimal  (less  than  one  in  more  than  100,000)  that  there 
is  no  difference  in  test  performance  on  Level  I  and  Level  II, 


Cumulative  Frequencies  and  Percentages  of  Scores  on  Level  II 
on  Men  Scoring  in  Each  of  Three  Groups  on  Level  I 
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TABLE  VIIIA 


Medians  and  Estimated  Standard  Deviations 
of  Scores  Made  by  Sub-Groups 

10/10  19/20-16/20  15/20  and  less 

Group  Group  Group 

Median  Score  17/20  13-5/20  10/20 

Standard  Deviation  4/20  4/20  4/20 


TABLE  VI TIB 

Chi-Sruares  of  iass-Fail  Cut  for  Each  Sub-Group 

10/10  I9/2O-I.6/2O  15/20  and  less 

Group  N  z  255  Group  M  : 122  Group  M  -  69 

Fredicted  Fails* **  152.9  73.6  39-3 

Obtained  Fails  .108  92  6? 

Chi-Square  39.1  13.7  25.7 

p  -  less  than  0.0000...  0.0000...  0.0000... 


*  On  basis  of  performance  of  whole  group  (N  -  446). 

**  Chi-Square  tost  shows  that  the  probability  that 
perron vinco  on  lev ■•■l  I  has  not  permitted  selection 
of  three  sub  groups  of  different  ability  in 
performance  on  hovel  II  are  less  than  1  in  more 
than  6000. 


Cmiulatlre  Percentage 
100 - 1 


Figure  2a 

Ciaulfttive  Percentage  Curves 
Plotted  on  Probability  Grid 


Scorea  on  Level  II 
Hade  by  Subgroups 


Group  scoring  10/10  on  I^*vel  I 

(H  I  255)  - 

Group  scoring  16/50  to  19/20  inclusive  on  Isvel  1 

(5  :  122)  - 

Group  scoring  15 /20  and  less 

(N  :  69)  . 
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